Identity recognition based on multiple feature fusion for an eye image

ABSTRACT

A method for identity recognition based on multiple feature fusion for an eye image, which comprises steps of registering and recognizing, wherein the step of registering comprises: obtaining a normalized eye image and a normalized iris image, for a given registered eye image, and extracting a multimode feature of an eye image of a user to be registered, and storing the obtained multimode feature of the eye image as registration information in a registration database; and the step of recognizing comprises:obtaining a normalized eye image and a normalized iris image, for a given recognized eye image,extracting a multimode feature of an eye image of a user to be recognized,comparing the extracted multimode feature with the multimode feature stored in the database to obtain a matching score, and obtaining a fusion score by fusing matching scores at score level, and performing the multiple feature fusion identity recognition on the eye image by a classifier. The present invention recognizes identity by fusing multiple features of eye regions on a human face, and thus achieves high recognition accuracy and is suitable for applications of high security level.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to pattern recognition and statisticsetc., more particularly to a method for multiple feature fusion personalidentity recognition based on eye images.

2. Description of Prior Art

Personal identity feature is the basic information of a human being,which is very important. However, it is hard for a knowledge- andmaterial-based identity recognition technology, such as password, secretcode, and ID card, to fulfill the requirements of large-scaleapplication and high level of security, and such technologies bringinconvenience to users. With the increasing development of intelligence-and information-based technologies in our society, a large-scaleidentify recognition technology has a great contribution on nationalsecurity, public security, economic security and network security.Biometrics technology is a technology for identity recognition by usingphysical and behavior features of a human being, and has advantages suchas high accuracy, high convenience for use, and high security. Widelyused existing biometric modes include face recognition, irisrecognition, voice recognition, fingerprint recognition, palm printrecognition, signature, and gait recognition etc. Correspondingbiometric systems are also successfully applied in various fields suchas access control and network security etc.

Most of the existing biometric technologies require a well cooperationby the users. For example, most of fingerprint and palm printrecognition devices are contact devices, while a non-contact devicerequires the user to corporate in a fixed manner. On one hand,inconvenience is introduced to the user, the system recognition rate isreduced, and the requirements of low response time and high flow volumein a large scale recognition scenario such as an air port, acustomhouse, or a station etc.); on the other hand, the system can onlyoperates in a passive recognition mode due to such well cooperation, inother words, the sensor can only receive data passively. However most ofthe security scenarios require an active recognition in which the sensormay obtain the information of the user with no or little corporation ofthe user. For example, it is desirable to authenticate the identity of aperson in the monitor scenario in real time without any corporate of theuser. Although some modes, such as face and gait, can be used inidentity recognition without any cooperation of the user, therecognition accuracy of face recognition and gait recognition is notsufficient to fulfill the practical requirements.

Human eye region contains pupil, iris, eyelid, periocular skin, eyebrow,eyelash, and etc. Iris has been proved to be one of the most accuratebiometric trait due to the high uniqueness of its texture. Irisrecognition systems have also been applied in public places like bank,customs, airport, coal mine, as well for the social affairs like welfaredistribution, missing children finding, and so on. In addition to iristexture, texture of periocular skin has good decidability and thus canbe used for identity recognition. In addition, iris and eye skin regionwill render a color characteristic under visible light, and thus can betaken as assistant features. For example, in additional to appearancefeature, the eye region has significant semantic features such asleft/right eye, double/single-edged eyelid, profile of eyelid, and soon, which may also be classified. Therefore, the eye region becomes abiometric trait with the best decidability due to its various features.

Besides the high uniqueness, biometric trait based on eye region is alsoa biometric trait which is easy to be used and populated. Eye is avisual organ for human to sense the world, so that the eye region isgenerally exposed to the outside. Even when the face is shield, eyeregion is still uncovered. Therefore the eye region is easy to becaptured by a visual sensor such as a camera. With the development ofoptical imaging technology, an active imaging system becomes possible.Related systems can acquire clear eye images from ten meters away oreven more. In view of the above, identity recognition based on eye imagecan achieve a user-friendly man-machine interaction and activerecognition functions.

Moreover, identity recognition based on eye region is very robust. Asingle modal biometric system is limited by application scenarios. Forexample, an iris texture suffering from disease is unable to be used iniris recognition. Because eye region contains multiple biometric traitmodes such as iris texture and skin texture, the multimode biometrictraits can be applied in various scenarios with few limitations.

In existing patents, a iris recognition system based on the uniquenessof iris is used for identity recognition by using eye regioninformation, in which other features of the eye is not used. Inaddition, all the existing patents related to iris recognition achieveidentity authentication by analyzing the local characteristic of iristexture feature of eye, such as the iris recognition algorithm proposedby Dr. John Daugman of University of Cambridge (U.S. Pat. No.5,291,560), in which feature coding is performed by a Cabor filter; themethod for iris recognition by analyzing shape feature of iris blobsproposed by Prof. Tieniu Tan et al. (CN 1684095). These methods arevulnerable to noise and rely on the accuracy of iris segmentation.

Sparse coding based iris recognition method in this patent is robust tothe environmental noises, and does not ask for an additional noisedetection based on image segmentation. Furthermore, in traditional scorelevel fusion, the affections by the distribution of scores and the datanoise is not considered, and thus the complementary characteristicbetween respective modes are not fully used.

SUMMARY OF THE INVENTION

Existing biometric recognition systems cannot simultaneously meet theneeds of high accuracy, high usability and high robustness due to theintrinsic drawbacks of the biometric traits, which significantly impedesthe widespread of biometric technology. This invention aims to realize abiometric recognition method with high accurate, less user cooperationand high applicability, based on the existing biometric traits andmultimode fusion technology.

The present invention achieves identity recognition by extractinginformation of different modes in different feature characterizingmanners and obtaining a plurality of matching scores in differentmatching manners, and obtaining a final fusion score via a robust scorelevel fusion strategy.

To achieve above objects, this invention provides a method for identityrecognition based on multiple feature fusion for an eye image, whichcomprises eye image preprocessing, multimode feature extraction,multimode feature matching, score level fusion and classification.

As described above, the method comprises steps of registering andrecognizing, wherein the step of registering comprises:

-   -   obtaining a normalized eye image and a normalized iris image,        for a given registered eye image, and    -   extracting a multimode feature of an eye image of a user to be        registered, and storing the obtained multimode feature of the        eye image as registration information in a registration        database, wherein the multimode feature includes an iris texture        feature, an iris color feature, an eye appearance feature, and        an eye semantic feature; and the step of recognizing comprises:    -   obtaining a normalized eye image and a normalized iris image,        for a given recognized eye image,    -   extracting a multimode feature of an eye image of a user to be        recognized, wherein the multimode feature includes an iris        texture feature, an iris color feature, an eye appearance        feature, and an eye semantic feature,    -   comparing the extracted multimode feature with the multimode        feature stored in the database to obtain a matching score, and        obtaining a fusion score by fusing matching scores at score        level, and    -   performing the multiple feature fusion identity recognition on        the eye image by a classifier.

In the present invention, identity is recognized by fusing variousfeatures in the eye region of the face of a human being, such that highsystem accuracy is achieved and the present invention is applicable inapplications requiring a higher security level. The present inventionreduces the cooperation of the user, and may be used as a long distanceactive identify recognition technology. The present invention isapplicable to both an eye image under visible light and an eye imageunder other monochromatic light.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1( a) shows a registering procedure of the method for identityrecognition based on multiple feature fusion for an eye image;

FIG. 1( b) shows a recognizing procedure of the method for identityrecognition based on multiple feature fusion for an eye image;

FIG. 2( a) shows a grayscale eye image;

FIG. 2( b) shows a color eye image;

FIG. 3 shows an eye image preprocessing procedure of the method foridentity recognition based on multiple feature fusion for an eye image;

FIG. 4 shows the results of iris region localization and eye regionlocalization for the eye image;

FIG. 5( a) shows a grayscale normalized eye image;

FIG. 5( b) shows a color normalized eye image;

FIG. 6( a) shows a grayscale normalized iris image;

FIG. 6( b) shows a color normalized iris image;

FIG. 7 shows a division of the color normalized iris image in extractingiris color feature;

FIG. 8 shows a division of the color normalized eye image in extractingeye appearance feature;

FIG. 9 shows a selection of the regions of interest and a configurationof the differential filters in extracting eye semantic feature;

FIG. 10( a) shows a training procedure of texton;

FIG. 10( b) shows a construction procedure of texton histrogram;

FIG. 11 shows an extraction procedure of eye semantic feature.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The object, scheme, and advantages of the present invention will beapparent, from the following description of the present invention bymeans of its specific embodiments with reference to the drawings.

The method for identity recognition based on multiple feature fusion foran eye image comprises steps of registering and recognizing.

Step of Registering R: As shown in FIG. 1( a), firstly the obtained eyeimage of a user to be registered is preprocessed (Step R0) to obtain anormalized eye image and a normalized iris image for feature extraction.Then a multimode feature of an image to be normalized is extracted bymeans of feature extraction, to obtain the registration information ofthe eye image and store it in a registration database. The Step ofRegistering R mainly comprises the following steps.

Step R0: The obtained eye image of a user to be registered ispreprocessed to obtain a normalized eye image and a normalized irisimage, in which the step of preprocessing comprises iris localization,iris normalization, eye region localization, and eye regionnormalization.

Step R11: After a grayscale normalized iris image of the eye image to beregistered is down-sampled, an iris texture feature vector v^(r)_(texture) is formed by arranging all the pixel values in line, andstored into an iris texture feature database.

Step R12: An iris color feature vector v^(r) _(color) of a colornormalized iris image of the eye image to be registered is extracted bymeans of color histogram, and stored into an iris color featuredatabase.

Step R13: An eye appearance feature vector v^(r) _(texton) of a colornormalized eye image of the eye image to be registered is extracted bymeans of eye texton histogram, and stored into an eye appearance featuredatabase.

Step R14: A eye semantic feature vector v^(r) _(semantic) of the colornormalized eye image of the eye image to be registered is extracted bydifferential filters and ordinal measuring, and stored into an eyesemantic feature database.

Step of Recognition S: As shown in FIG. 1( b), firstly the obtained eyeimage of a user to be recognized is preprocessed to obtain a normalizedeye image and a normalized iris image for feature extraction. Then amultimode feature is extracted by means of feature extraction. Then, theobtained feature is compared with the feature in the database by meansof specific matching, to calculate a comparing score. A final matchingscore is obtained by fusing at score level. As shown in FIG. 1( b), theStep of Recognition S mainly comprises the following steps.

Step S0: The obtained eye image of a user to be recognized ispreprocessed to obtain a normalized eye image and a normalized irisimage, in which the step of preprocessing comprises iris localization,iris normalization, eye region localization, and eye regionnormalization.

Step S1: A multimode features of the eye images is extracted byfollowing steps:

Step S11: An iris texture feature vector v^(s) _(texture) of thenormalized iris image to be recognized is extracted by means of sparsecoding;

Step S12: An iris color feature vector v^(s) _(color) of the normalizediris image to be recognized is extracted by means of color histogram;

Step S13: An eye appearance feature vector v^(s) _(texton) of thenormalized eye image to be recognized is extracted by means of eyetexton histogram;

Step S14: An eye semantic feature vector v^(s) _(semantic) of thenormalized eye image to be recognized is extracted by differentialfilters and ordinal measure.

Step S2: Matching between multimode feature vectors is performed byfollowing steps:

Step S21: A comparing score S_(texture) is calculated as areconstruction error between the iris texture feature vector v^(s)_(texture) of the recognized image and the iris texture feature vectorof each class of registered image in the database;

Step S22: A comparing score S_(color) is calculated as the Euclideandistance distance between the iris color feature vector color v^(s)_(color) of the recognized image and the iris color feature vector v^(r)_(color) of the registered image in the database;

Step S23: A comparing score S_(texton) is calculated as the Euclideandistance between the eye appearance feature vector v^(s) _(texton) ofthe recognized image and the eye appearance feature vector v^(r)_(texton) of the registered image in the database;

Step S24: A comparing score S_(semantic) is calculated as the XORdistance between the eye semantic feature vector v^(s) _(semantic) ofthe recognized image and the eye semantic feature vector v^(r)_(semantic) of the registered image in the database.

Step S3: The multimode comparing scores are fused. A final comparingscore S′_(f) is obtained by adaptively fusing at score level.

Step S4: Classifying by a Nearest Neighborhood classifier.

The key steps related to the method of the present invention will bedescribed in detail. Particularly, respective basic steps of the presentinvention are described as follows.

Preprocessing of Image

During either a registration process or a recognition process, it isnecessary to apply iris localization (as shown in FIG. 4) to thecaptured original eye image (as shown in FIG. 2( b)) and then obtain anormalized eye image (as shown in FIG. 5) and a normalized iris image(as shown in FIG. 6) which may be used for feature extraction. As shownin FIG. 3, particular steps are described as follows.

Iris localization is performed. Two circles are used respectively to fitthe internal and external boundaries of the iris in the capturedoriginal eye image (as shown in FIG. 2( b), a RGB image of a resolutionof 640×480), i.e. the boundary between iris and pupil and the boundarybetween iris and sclera (the iris localization result image of thepreprocessed original iris image as shown in FIG. 4). The irislocalization may be implemented by the integral-differential operator asdescribed in the U.S. Pat. No. 5,291,560, in which a iris localizationresult is obtained by applying integral-differential operation on thegrayscale image (as shown in FIG. 2( a)). The integral-differentialoperator is expressed as follows:

$\underset{({r,x_{0},y_{0}})}{argmax}{{{G_{\sigma}(r)}*\frac{\partial}{\partial r}{\oint_{({r,x_{0},y_{0}})}{\frac{I\left( {x,y} \right)}{2\pi \; r}{s}}}}}$

where G_(σ)(r) represents a Gaussian function with a variance σ. I(x, y)represents the eye image, and (r, x₀, y₀) represents parameters of acircle. Integral-differential operator is a circle boundary detector,the basic principle of which is to find a boundary defined along suchparameters in the parameter space (r, x₀, y₀) of the circle, perform adifferential operation and then an integral operation, normalize theresulting values based on the perimeter of the circle to obtain anintegral-differential energy value on such parameters, and use theparameter having a highest energy value in the parameter space as afinal detected circle. In the iris image, the boundary between iris andpupil and the boundary between iris and sclera are both circle shaped,generally employ the two higher parameters among theintegral-differential values, and are distinguished in accordance withthe radius of the circles: the one with a smaller radius is consideredas the localization result of the boundary between iris and pupil, andthe one with a larger radius is considered as the localization result ofthe boundary between iris and sclera (as shown in FIG. 4, the irislocalization results are denoted by two red circle regions).

Normalized iris image is obtained. The iris size varies betweendifferent iris images and the iris will enlarge or shrink as thevariation of light. Therefore it is necessary to normalize the irisregions of different sizes before feature extraction. Since the internaland external boundaries of iris are obtained from iris localization, theiris normalization can be performed based on a “rubber sheet” modelprovided in U.S. Pat. No. 5,291,560. The basic principle is to normalizethe circle iris region in the original image into a rectangular colornormalized iris image of a fixed size (as shown in FIG. 6( b), a colorRGB image of a resolution of 512×66) based on a transformation fromCartesian coordinate system to polar coordinate system, and perform animage mapping in a linear straight in manner. Sampling in angulardirection is performed clockwise from zero degree in horizontaldirection (as shown in FIG. 6( b), the direction indicated by the redarc arrow), and sampling in radial direction is performed from theboundary between iris and pupil (as shown in FIG. 6( b), the directionindicated by the red radial arrow). Then the red channel is chosen asthe grayscale normalized iris image (as shown in FIG. 6( a), of aresolution of 512×66). Normalized eye image is obtained. The center ofiris (as shown in FIG. 4) is determined in the captured original eyeimage (as shown in FIG. 1, a RGB image of a resolution of 640×480). Arectangular region of a fixed size (as shown in FIG. 4, the rectangularregion having a center the same as the center of iris, a length fourtimes larger than the radius of iris, and a width three times largerthan the radius of iris) is defined as an eye region of interest,depending on a fixed center position of eye and the radius R of iris inthe normalized image. The eye region of interest is mapped to a regionof a fixed size in a bilinear manner, as the normalized eye image. A redchannel of a color normalized eye region (as shown in FIG. 5( b), of aresolution of 200×150) is employed as a grayscale normalized eye image(as shown in FIG. 5( a), of a resolution of 200×150).

Extraction of Feature

After the normalized iris image and normalized eye image are obtained,extraction of feature is desired. Particular features as used hereininclude iris texture feature based on sparse coding, iris color featurebased on color histogram, eye appearance feature based on SIFT textonhistogram, and eye semantic feature based on differential filter andordinal measurement. Particular steps of extraction are described asfollows.

Iris texture feature is a basic feature of the iris.

High recognition rate is achieved, since a feature representation basedon sparse coding is capable of overcome the influence by noise such asshielding in face recognition. Therefore the extraction of iris texturefeature also uses sparse coding. The basic principle of sparse coding isthat: all the samples of one class may be obtained by linear combiningseveral samples of its own. Given m classes of iris with each class icomprises n registration samples. All the registration samples constructa group: X: {x₁₁, x₁₂, . . . , x_(1n), . . . , x_(ij), . . . , x_(m1),x_(m2), . . . , x_(mn)}. Given a sample y of the iris to be recognized,optimization is performed in accordance with α*=arg min∥α∥₁ under aconstraint condition Xα=y. The registration sample x_(ij) is a iristexture feature vector v^(r) _(texture) to be registered, and iscomposed of grayscale pixel values of a grayscale normalized iris imageto be registered which is downsampled at a ratio of 0.25. Theregistration sample x_(ij) has a size of 512×66/4=8448. y is a sample tobe recognized, and is composed of grayscale pixel values of a grayscalenormalized iris image to be recognized which is downsampled at a ratioof 0.25. y has a size of 8448. The optimization problem is solved toobtain a solution: α*: {α*₁₁, α*₁₂, . . . , α*_(1n), . . . , α*_(ij), .. . , α*_(m1), α*_(m2), . . . , α*_(mn)} as the iris texture featurevector v^(s) _(texture) for recognition, with the size of the solutionbeing m×n.

Iris renders a color feature under visible light. Color histogram iscommonly used to represent the color feature. The color feature has alocal area characteristic, in which different regions under visiblelight have different color distributions. Accordingly, regional colorhistogram is more suitable to represent the color feature of iris. RGBcolor eye image is transformed into 1αβ color space, and the colorfeature of iris is extracted in 1αβ color space. For example, the colornormalized iris image is divided into 3×1 blocks, i.e., is divideduniformly into 3 blocks along vertical direction (as shown in FIG. 7),in which each block is of a size of 22×512. The color histogram isconstructed by calculating the frequency at which each color valueoccurs on each of the three channels 1, α, β, in which the size of thecolor space of each channel is 256. Finally, all the 9 color histogramsof 3 channels and 3 blocks are concatenated into a color histogram, asan iris color feature vector v^(s) _(color) having a dimension of256×3×9=6912. The transformation from RGB space to 1αβ space is asfollows:

$\begin{bmatrix}L \\M \\S\end{bmatrix} = {{{\begin{bmatrix}0.3811 & 0.5783 & 0.0402 \\0.0606 & 0.3804 & 0.0453 \\0.0241 & 0.1228 & 0.8444\end{bmatrix}\begin{bmatrix}R \\G \\B\end{bmatrix}}\begin{bmatrix}l \\\alpha \\\beta\end{bmatrix}} = {{\begin{bmatrix}\frac{1}{\sqrt{3}} & 0 & 0 \\0 & \frac{1}{\sqrt{6}} & 0 \\0 & 0 & \frac{1}{\sqrt{2}}\end{bmatrix}\begin{bmatrix}1 & 1 & 1 \\1 & 1 & {- 2} \\1 & {- 1} & 0\end{bmatrix}}\begin{bmatrix}{\log \; L} \\{\log \; M} \\{\log \; S}\end{bmatrix}}}$

In additional to the iris features, the appearance feature of the entireeye region also has a certain distinction. The eye texture and skintexture are used as an uniform eye appearance feature in identityrecognition. Texton histogram is an effective texture analysingapproach, whose basic principle is that: a texture pattern is composedof basic elements (textons), and the pattern varies with thedistribution of the basic elements. FIG. 10( a) shows the procedure oftexton acquisition. Firstly, a feature vector of each pixel on eachchannel of all the normalized color eye image used for training isextracted by using an SIFT local descriptor, with the dimension of thefeature vector is 128. The all of the obtained local feature vectors areclustered into K groups via K-means clustering, with the center of eachgroup taken as texton, and all the textons construct a textondictionary. Moreover, three dictionaries are constructed for R, G, Bchannels. FIG. 10( a) shows the procedure of appearance featureextraction of eye region. Each color normalized eye image is uniformlydivided into 2×2=4 blocks, and then SIFT local feature vector isextracted on each pile of each color channel within each block. Afterthat, each local feature vector is quantized into a texton having aminimum distance (generally Euclidean distance) from the local featurevector. The frequency at which each texton occurs is calculated toconstruct a texton histogram for this region on each channel. Finally,all the texton histograms (3 channels×4=12) are concatenated into theeye appearance feature vector v^(s) _(texton) of a size of k×12. Thecalculation procedure of SIFT can be found in U.S. Pat. No. 6,711,293.

A label of left or right eye is used to mark the eye semantic feature.0/1 is used to mark left/right eye. In particular, the left/right eye ismarked in accordance with the difference between the distribution of theeyelashes on the left side of the upper eyelid and the distribution ofthe eyelashes on the right side of the upper eyelid. The eyes are markedwith numbers by comparing the eyelash density of a part near the outercanthus and the eyelash density of a part away from the tear duct. Giventhe color normalized eye image, the extraction procedure of eye semanticfeature is shown in FIG. 11 comprising eyelid fitting, selection of theregion of interest (ROI), filter design, eyelash density estimation, andsemantic feature encoding.

Eyelid fitting: Firstly, boundary information is obtained by using aCanny boundary detector in a grayscale normalized eye image. Then basedon the result of iris localization, boundary points of the upper-leftand upper-right regions of the outer circle of iris are sleeted to bestraight line fitted, to obtain a coarse fitting of the upper eyelid (asshown in FIG. 9 by the white lines).

Selection of the region of interest: Based on the result of eye fitting,a ROI is selected. As shown in FIG. 9, the black rectangle boxesrepresent the selected ROIs. Selection of the right ROI is taken as anexample. O_(I) is the center of outer boundary of iris region, and R isthe radius. L_(R) is an intersection point of the diameter L^(R)O_(I) invertical direction and the right line eyelid fitting straight lineL_(R)E_(R). The length of L_(R)E_(R) is R, and M_(R) is the center ofL_(R)E_(R). The black box labels the selected ROI with a width R and aheight R/2. The long side is parallel to L_(R)E_(R). Selection of theleft ROI is achieved in a similar manner.

Filter design: A left and right differential filters are designed forleft and right ROIs respectively. Take the right filter as an example.The right filter has the same size and direction as those of the rightROI. As shown in FIG. 9, all the red regions are set to 1, and the blankportion is set to −1. The directions of the differential filters arevertical to the straight line on the right side of the eyelid, toachieve adaptive direction. The length of the filters are set to have alength of R and a width of R/2 in accordance with the radius of theiris, to achieve adaptive size, in which R represents the radius of thecircle obtained by fitting the external boundary of iris. The leftfilter is designed similarly.

Eyelash Density Estimation: Taking the right side as an example, theeyelash density of ROI is estimated in the color normalized eye image.Convolution is achieved by using the right filter and the right ROIregion on each of the color channels respectively, to obtain a responseon each channel. The resulting responses on R, G, B channels are summedto obtain a final result as the eyelash density estimation DR of theright ROI. The eyelash density estimation DL of the right ROI iscalculated similarly.

Semantic Feature Encoding: An eye semantic feature vector v^(s)_(semantic) is generated by ordinal measuring. If D_(L)>D_(R), v^(s)_(semantic)=1, otherwise, v^(s) _(semantic)=0.

Matching Strategy:

In recognition, matching between the feature vector to be recognized andthe registered feature vector is desired. Based on the above described 4types of features, matching of the iris semantic feature is implementedbased on logical XOR distance, i.e. S_(semantic)=XOR(v^(s) _(semantic),v^(r) _(semantic)). The iris color feature and the eye appearancefeature are implement based on Euclidean distance as follows:

S=d(ν₁, ν₂)=∥ν₁−ν₂∥₂

For the iris texture feature, given a sample y for recognition and aregistration sample feature vector x: {x₁₁, x₁₂, . . . , x_(1n), . . . ,x_(ij), . . . , s_(m1), x_(m2), . . . , x_(mn)}, the iris texturefeature vector v^(s) _(texture) for recognition is obtained by sparsecoding, i.e. α*: {α*₁₁, α*₁₂, . . . , α*_(1n), . . . , α*_(ij), . . . ,α*_(m1),α*_(m2), . . . , α*_(mn)}. The reconstructed recognition sampley* is obtained by using a reconstruction coefficient α*_(i): {α*_(i1),α*_(i2), . . . , α*_(in)} of each class and a sample X_(i): {x_(i1),x_(i2), . . . , x_(in)}, and the matching scores between the sample andall the samples of each class are recognized as a reconstruction erroras following:

S _(i)=∥y−y_(i)*∥₂ , y*=α* _(i) X _(i)

It will be noted that, the smaller the values of the above four types ofcomparing scores are, the smaller the difference between theregistration feature and the recognition feature is.

After the matching scores for the four types of features, i.e.S_(texture), S_(color), S_(texton), S_(semantic) a final comparing scoreis obtained by a score-level adaptive fusion strategy. The adaptivescore-level-fusion strategy comprises: score normalizing, weightedsumming, and adaptive adjusting.

Score normalizing: Before score fusing, it is desirable to normalize theoriginal four types of comparing scores into the same range [0, 1]. Thesmaller the value of the comparing score is, the smaller the differencebetween the registration feature and the recognition feature is. Thereare many methods of score normalizing in documents, in which the“min-max” normalizing method is the simplest and effective one. Given aset of matching scores S: {s₁, s₂, . . . , s_(n)}, the “min-max”normalizing is implemented as follows:

$s_{i}^{\prime} = \frac{s_{i} - {\min (S)}}{{\max (S)} - {\min (S)}}$

The original four matching scores are normalized to obtained fournormalized scores S′_(texture), S′_(color), S′_(texton), S′_(semantic).

After normalizing, score level fusion is performed by a weighted summingstrategy, to obtain a fusion comparing score as follows:

S _(f) =w ₁ S′ _(texture) +w ₂ S′ _(color) +w ₃ S′ _(texton),

where w_(i) is the weight, (i=1, 2, 3, w₁+w₂+w₃=1). Generally w₁ has thesame value, meaning that each feature has the same weightiness.

To eliminate the influence of noise, the fusion comparing score may berevised in accordance with the eye semantic feature matching score, toobtain a revised fusion score S′_(f). The rules for revising is asfollows:

if S′_(semantic)=1 and S_(f)<M₁, S′_(f)=M₁

if S′_(semantic)=0 and S_(f)>M₂, S′_(f)=M₂, M₁>M₂

The first rule means that: if the semantic feature of the recognitioneye image is different from that of the registration eye image and thefusion result is that the recognition eye image is similar to theregistration eye image, the original fusion score is enlarged to M₁,trending to make the recognition eye image not similar to theregistration eye image. The second rule means that: if the semanticfeature of the recognition eye image is different from that of theregistration eye image and the fusion results of the other three typesof features shows that the recognition eye image is not similar to theregistration eye image, the original fusion score is reduced to M₂,trending to make the recognition eye image not similar to theregistration eye image.

The classifier of the present invention is a Nearest Neighborhoodclassifier, namely the class of identity with the smallest matchingscore is the final recognized identity.

Embodiment 1: Application of the Method for Identity Recognition Basedon Multiple Feature Fusion for an Eye Image in a Network TransactionPlatform

The present invention may be widely applied in identity recognition in anetwork platform based on a network camera. As the development ofelectronic commerce, transaction on network platform is introduced intoour social life. At the same time, cheat accompanies. The security levelof traditional identity recognition modes based on secret code andpassword is unlikely to fulfill practical requirements. Biometricstechnology becomes an effective solution. Identity recognition based oneye image has great contributions. When a user is registering, eyeregion information is transmitted to a third party authentication centerby a normal network camera. The remote authentication center willregister the biometric trait of the user into a system database by usingthe registration algorithm of the present invention. When the user isperforming a network platform identity authentication, the networkcamera will transmit the captured eye region information to the thirdparty authentication center. The remote authentication center will referto the system database by using the recognition algorithm of the presentinvention, to perform identity authentication. Such method may achieve aconvenient and effective identity authentication, and thus ensure thesecurity of the personal identity information on the network platform.

Embodiment 2: Application of the Method for Identity Recognition Basedon Multiple Feature Fusion for an Eye Image in a Security MonitorScenario

The present invention may be widely applied in a security monitorscenario. In the security monitor scenario, it is desirable to monitorthe people occurring in the scenario, such that alarming will betriggered in good time when illegal people occurs. For example, eyeregion information of a criminal who has been arrested is registered inthe system database, to prevent him from committing a crime in thefuture. When a criminal occurs in the capture range of the networkmonitor camera, his/her eye region information will be transmitted to aprocessing terminal via network, and the processing terminal willdetermine his/her identity by using the recognition algorithm of thepresent invention. If he/she is determined to be a criminal, alarmingwill be triggered in good time so as to arrest the criminal.

Although illustrative embodiments have been described herein in detail,the scope of the present invention is not limited thereto. Those skilledin the art will appreciate that variations and substitution may be madewithout departing from the spirit and scope of this invention. Thereforethe scope of the present invention is defined by the claims.

1. A method for identity recognition based on multiple feature fusionfor an eye image, which comprises steps of registering and recognizing,wherein the step of registering comprises: obtaining a normalized eyeimage and a normalized iris image, for a given registered eye image, andextracting a multimode feature of an eye image of a user to beregistered, and storing the obtained multimode feature of the eye imageas registration information in a registration database; and the step ofrecognizing comprises: obtaining a normalized eye image and a normalizediris image, for a given recognized eye image, extracting a multimodefeature of an eye image of a user to be recognized, comparing theextracted multimode feature with the multimode feature stored in thedatabase to obtain a matching score, and obtaining a fusion score byfusing matching scores at score level, and performing the multiplefeature fusion identity recognition on the eye image by a classifier 2.The method according to claim 1, wherein the multimode feature includesan iris texture feature, an iris color feature, an eye appearancefeature, and an eye semantic feature.
 3. The method according to claim1, further comprises image preprocessing, which comprises iris imagepreprocessing and eye image preprocessing.
 4. The method according toclaim 3, wherein iris image preprocessing includes iris localization andiris image normalization.
 5. The method according to claim 4, whereiniris localization is implemented by double-circle fitting, in which twocircles are used to fit the boundary between iris and pupil and theboundary between iris and sclera respectively.
 6. The method accordingto claim 4, wherein the iris image normalization comprises: mapping theoriginal circle iris region into a rectangular region of a fixed size bytransforming from a Cartesian coordinate system to a polar coordinatesystem.
 7. The method according to claim 3, wherein eye imagepreprocessing includes eye region localization and eye regionnormalization.
 8. The method according to claim 7, wherein eye regionlocalization comprises: labeling eye region as a rectangular regionhaving a center identical to the center of a circle which is obtained byfitting the boundary between iris and sclera.
 9. The method according toclaim 7, wherein eye image normalization comprises: scaling the originaleye region into a rectangular region of a fixed size.
 10. The methodaccording to claim 2, wherein the iris texture feature is extracted bysparse coding.
 11. The method according to claim 2, wherein the iriscolor feature is extracted by a color histogram.
 12. The methodaccording to claim 2, wherein the eye appearance feature is extracted bytexton representation.
 13. The method according to claim 2, wherein theeye semantic feature is extracted by a differential filter and anordinal measuring characteristic.
 14. The method according to claim 11,wherein the iris color feature vector is obtained by extracting a colorhistogram on each of color channels of each block of image andconcatenating all the color histograms.
 15. The method according toclaim 11, wherein color histogram is extracted in 1αβ color space. 16.(canceled)
 17. The method according to claim 16, wherein extracting theeye appearance feature by a texton histogram comprises: extracting alocal feature for each pixel of each image via scale invariant featuretransformation; constructing a texton dictionary by K textons which areobtained via K-means clustering; constructing texton histograms, bydividing the normalized iris eye image into non-overlap blocks andcalculating the frequency at which each texton occurs in accordance withthe matching degree between a local descriptor and the texton for eachblock; and concatenating the texton histograms of all the block regionsto form the eye appearance feature vector.
 18. The method according toclaim 17, wherein if each divided image is a color image, the frequencyat which each texton occurs is calculated on each color channel.
 19. Themethod according to claim 13, wherein the extracting the eye semanticfeature comprises: obtaining a coarse position of eyelash region ofupper eyelid in accordance with the result of iris localization andeyelid localization; choosing a left part and a right part of theeyelash region of upper eyelid as regions of interest, based on theposition of eyelash region of upper eyelid and iris region; generating aleft and right differential filters having adaptive sizes and directionsbased on the radius of iris and the direction information of uppereyelid; obtaining an estimation of the density of the left region ofinterest by convoluting the left differential filter and the left regionof interest, and obtaining an estimation of the density of the rightregion of interest by convoluting the right differential filter and theright region of interest; generating the eye semantic feature vector inaccordance with an ordinal measuring characteristic based on theestimations of the densities of the left and right regions of interest.20. The method according to claim 18, wherein the adaptive differentialfilters are configured such that the upper eyelid is fitted by a leftand right straight lines, the directions of the left and rightdifferential filters are perpendicular to the left and right straightlines respectively to achieve adaptive direction, and the length andwidth of each filter are set depending on the radius of iris to achieveadaptive size.
 21. The method according to claim 18, wherein if the eyeimage is a color eye image, the regions of interest of the eye image onthe R, G, B channels are convoluted by the filters, and the convolutionvalues on all the channels are summed to obtain a final response valuefor the filters and the regions of interest of the color eye image. 22.The method according to claim 1, wherein fusing at score levelcomprises: normalizing the matching score S_(color) of the iris colorfeature, the matching score S_(texture) of the iris texture feature, andthe matching score S_(texton) of the eye appearance feature inaccordance with the distributions of their original scores respectively,to obtain normalized scores S′_(color), S′_(texture), S′_(texton);weighted summing the normalized scores S′_(color), S′_(texture),S′_(texton), S′_(texton) to obtain the fusion scoreS_(f)=w₁S′_(texture)+w₂S′_(color)+w₃S′_(texton), revising the fusionscore S_(f) obtained by weighted summing, based on the comparison resultS′_(semantic) of the eye semantic feature, to obtain S′_(f), where w₁represents the weight, (i=1, 2, 3, w₁+w₂+w₃=1).
 23. The method accordingto claim 22, wherein the revising follows a rule resulting from thetrending of the fusion score and the eye semantic feature, in which ifthe eye semantic feature is similar while the other features are notsimilar, then the fusion score is revised such that the eye semanticfeature is changed to be more similar; if the eye semantic feature isnot similar while the other features are similar, then the fusion scoreis revised such that the eye semantic feature is changed to be lesssimilar.